Using Multivariate Testing To Select The Best Subject Line For A Mailing

You have a brief to deploy an email to 10,000 recipients from your customer database and your creative team has two possible subject lines. Use multivariate testing to try each one on a control group and see which one results in the most emails being opened. You will then use the winning subject line for the bulk of your mailing.

Tactics Tab

Configure a Send a Message tactic as required. The Email Subject line will be a campaign variable (in this example it is called ABTest_1).

Audience Tab

Configure your Audience as required.

Variables Tab

Create the variable that will provide the Subject Line text:

• Propagation Level: CampaignRecipient
• Name: ABTest_1
• Type: Text
• Define As: Detail
• Default Value: Select literal values and enter the Subject Line options for example 'SubjectLine1', 'SubjectLine2' and 'Deployment'.

Tracks Tab

The example has three tracks:

• Start track contains the initial split into three groups and the deployment to the two control groups.
• Main Deployment track contains the multivariate test on the control groups and the subsequent deployment to the main audience using the most successful subject line.
• Done track is where recipients will be sent once they have been processed.

Start Track

Procedure

1. SubTest1, Proportional checked, Value 10
2. SubTest2, Proportional checked, Value 1
3. Deploy, Proportional checked, Value 80

4. In SubTest1 branch:
   1. Set ABTest_1 variable to 'SubjectLine1'
   2. Implement Tactic 1
5. In SubTest2 branch:
   1. Set ABTest_1 to 'SubjectLine2'
   2. Implement Tactic 1
6. In the Deploy branch, do nothing
7. Add a Go to Track to send the remainder of recipients to the MainDeployment track.

Main Deployment Track

Procedure

1. Remove the default Do Immediately time item and add an After a Period node.
2. Set this to After 2 days which would be a suitable lapse to test the open rates of the two control groups.
3. Drag the AB Testing tool into the After a Period node.
4. Click into the grey event selection area and select the Email Opened event.
5. Drag in the Campaign Variable tool or select it from the Create new drop-down. Enter the variable SubjectLine1 resolved to the Customer table.
6. Drag in another Campaign Variable tool and enter the variable Subjectline2 also resolved to the Customer table.

You should now have two When matches branches below for the two Subject Line variables.

7. Set variables and implement the tactic for each branch:
   • In SubjectLine1 branch:
     • Add a Set Variable tool and set ABTest_1 to SubjectLine1
     • Implement Tactic 1
   • In SubjectLine2 branch:
     • Add a Set Variable tool and set ABTest_1 to SubjectLine2
     • Implement Tactic 1
8. Send all recipients to the Done track by adding a Go to Track at the end of the MainDeployment track. The Tracks tab should now look like this:

Done Track

There is nothing for you to do here.

• AB Testing (Multivariate Segment)
  • Using Multivariate Testing to Select the Best Subject Line for a Mailing
  • Training video - AB testing